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DOMESTIC  VIOLENCE  TRAINING:  STRATEGY  AND  TACTICS 

INTRODUCTION 

The  Commonwealth  of  Massachusetts  has  focused  an  increasing  amount  of  attention  on 
programs  aimed  at  diminishing  domestic  violence  and  has  disseminated  resources  to  these 
programs  accordingly.  Such  programs  have  included  those  designed  to  prevent  the  recurrence  of 
domestic  violence  by  training  law  enforcement  officers  on  how  to  respond  to  related  incidents. 
The  federal  Edward  Byrne  Memorial  Formula  Grant  program  in  Massachusetts  has  identified 
domestic  violence  programs  with  training  and  education  components  as  priorities  among 
recipients  of  funding.  Training  for  law  enforcement  and  criminal  justice  officials,  in  general,  has 
been  emphasized  by  a  number  of  state  funded  initiatives  in  the  Commonwealth  as  well  as  many 
other  states. 

The  allocation  of  funding  to  training  programs  was  supported  by  the  belief  that  such 
programs  can  be  the  much  needed  link  between  officer  behavior  and  the  reduction  of  violence  in 
domestic  disputes,  whether  against  officers  and/or  victims  (C.J.  of  the  Americas,  1991;  Crime 
Control  Digest.  1991a;  Training  Aid  Digest.  1991).  Court  decisions  have  also  underscored  the 
importance  of  adequate  training  (AELE  Liability  Reporter,  1990;  Crime  Control  Digest.  1991b). 
In  addition,  there  has  been  some  suggestion  that  the  failure  to  provide  training  in  an  area  may  be  a 
way  of  avoiding  the  problem  (Aaron,  1991). 

Training  programs  for  law  enforcement  personnel  who  respond  to  domestic  violence  calls 
use  several  tactics.  Primary  objectives  of  the  trainings  include  conveying  information,  providing 
moral  and  administrative  support,  and  the  opportunity  for  role-playing  different  responses 
(Massachusetts  Criminal  Justice  Training  Council  [MCJTC],  1992).  Each  of  these  methods  were 
considered  necessary  for  changing  knowledge,  attitudes,  and  behavior  of  criminal  justice 
personnel  (MCJTC,  1992).  As  with  any  program  intended  to  invoke  change  in  how  people  handle 
a  situation,  training  programs  must  carefully  address  appropriate  issues  and  consider  the  unique 
aspects  of  both  the  intended  audience  and  the  material  presented  to  be  effective.  If  training 
materials  are  improperly  designed,  they  may  even  encourage  violence  (Crime  Control  Digest. 
1991a). 

Even  when  training  programs  appear  to  change  participants'  attitudes  or  knowledge,  they 
do  not  necessarily  alter  behavior  (Braye  &  Preston-Shoot,  1991;  Rynerson  &  Fishel,  1993;  Sgarzi, 
1992).  However,  such  changes  have  been  considered  to  at  least  encourage  desired  behavioral 
changes  (Gagnon  &  Lavoie,  1990;  Stith,  1990).  Thus,  when  evaluating  training  programs,  one 
must  examine  which  strategies  and  tactics  are  most  likely  to  produce  desired  outcomes.  This 
report  describes  the  evaluation  of  two  domestic  violence  law  enforcement  training  programs  in 
relation  to  the  enhancement  of  increasing  both  knowledge  in  the  area  and  related  changes  in 
officer's  behavior. 
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This  report  provides  an  overview  of: 

0  The  process  undertaken  to  conduct  the  evaluation; 

0  The  methodology  and  major  findings  of  the  evaluation; 

0  Obstacles  to  the  study  and  strategies  for  dealing  with  them;  and 

0  A  description  of  the  process  of  integrating  evaluation  findings  into  programs. 

Problems  Evaluating  Training 

Evaluating  the  implementation  and  impact  of  these  programs  has  posed  a  challenge  for  the 
Commonwealth's  Statistical  Analysis  Center,  the  Massachusetts  Committee  on  Criminal  Justice, 
and  evaluators  in  other  states  (Brown,  1990;  Choi,  1990;  Dunworth  &  Saiger,  1992;  Gabel  & 
Nimick,  1990;  Garner  and  Visher,  1988;  Holmes,  Mignon  &  Headley,  1993;  Urquiza,  1991). 
This  challenge  arises  because  of  the  difficulties  in  measuring  implementation  and  impact,  as  well 
as  problems  in  design.  It  is  difficult  to  develop  appropriate  measures  of  changes  in  knowledge, 
attitudes,  and  behaviors  intended  to  be  modified  by  these  training  programs.  There  are  also 
difficulties  in  measuring  the  effects  of  presumed  changes  in  the  knowledge,  attitudes,  and 
behaviors  of  criminal  justice  officials  on  domestic  violence  outcomes.  Some  of  the  training 
programs  also  state  expectations  for  "long-term  outcomes,"  rather  than  short  term  results  more 
amenable  to  study. 

The  designs  used  are  often  compromised  by  legal,  ethical,  political,  and  resource 
constraints.  Even  when  a  powerful  and  rigorous  experimental  design  is  used,  the  protocol  may  be 
vulnerable  to  violation.  Some  of  the  controversy  over  the  Minneapolis  domestic  violence 
experiment  resulted  from  officer  ability  to  change  the  randomly  assigned  treatments  (Sherman, 
1993).  When  the  experimental  protocol  breaks  down,  it  is  extremely  difficult  to  redesign  the 
study  to  substitute  an  alternative  powerful  design.  Resources  have  already  been  expended. 
Promises  to  research  sites,  Institutional  Review  Boards,  and  subjects  have  already  been  made. 
Credibility  and  trust  may  also  be  compromised. 

Problems  in  evaluating  these  programs  raise  questions  that  need  to  be  addressed.  What 
are  the  expected  outcomes  of  these  programs?  Is  it  to  change  knowledge,  feelings,  attitudes, 
behavior,  or  some  combination  of  the  above?  How  can  these  changes  be  measured  in  ways  that 
have  reasonable  levels  of  validity  and  reliability?  What  research  designs  can  provide  information 
on  the  outcomes  of  these  projects  that  meet  acceptable  professional  standards  for  evaluation 
information? 

To  answer  these  questions  this  project  has  three  specific  goals: 

0        Improving  the  capacity  of  the  Commonwealth  of  Massachusetts  to  evaluate 

criminal  justice  training  programs,  including  improved  measures  of  the  impact  of 
domestic  violence  training  programs; 
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0        Production  of  a  report  identifying  more  successful  domestic  violence 
training  programs  in  Massachusetts; 

0        Production  of  a  report  identifying  procedures  and  strategies  for  enhancing 
capacity  of  state  agencies  to  evaluate  criminal  justice  training  programs. 

In  achieving  these  goals,  the  above  questions  are  addressed. 

Addressing  these  problems  for  evaluating  domestic  violence  training  programs  is 
important  because  of  the  human  costs  of  the  violence.  This  is  also  a  priority  subject  area  upon 
which  federal  intervention  is  intended  to  have  an  impact  (National  Institute  of  Justice  [NIT], 
1993).  Reducing  these  problems  increases  the  capacity  of  state  agencies  to  better  evaluate 
criminal  justice  training  programs,  in  general,  and  domestic  violence  training  programs,  in 
particular.  It  also  contributes  to  research  evaluating  domestic  violence  intervention  programs,  for 
which  measurement  issues  have  been  a  recognized  problem  (Sherman,  1993).  Findings  from  this 
study  aid  current  Department  of  Justice  program  initiatives  in  domestic  violence  (Andrew,  1989; 
NU,  1993). 

Previous  and  Current  Work 

The  Statistical  Analysis  Center  (SAC)  in  Massachusetts  has  conducted  previous  research 
in  domestic  violence  and  the  evaluation  of  domestic  violence  intervention  programs  (Holmes  & 
Bibel,  1989;  Holmes,  et  al.,  1993).  Information  from  these  studies  provided  some  limited 
evidence  of  positive  effects  of  training  programs.  The  utility  of  the  evidence  was  reduced  because 
the  findings  were  unable  to  address  some  issues  for  which  measurement  problems  could  not  be 
resolved.  The  studies  strongly  identified  a  need  to  improve  measurement  of  key  variables  when 
evaluating  these  programs. 

EVALUATION  OF  TWO  DOMESTIC  VIOLENCE  TRAINING  PROGRAMS 

Program  Selection  for  Evaluation 

All  six  of  the  domestic  violence  programs  funded  by  the  Massachusetts  Committee  on 
Criminal  Justice  (MCCJ)  with  money  from  the  federal  Byrne  grant  were  originally  considered  for 
this  evaluation  project  at  the  inception  of  the  study.  Five  of  these  projects  had  training 
components:  the  Roxbury  Court  Domestic  Violence  Program,  the  Cambridge  Police 
Department's  Operation  Safe  Home,  the  Criminal  Justice  Training  Council  programs  on  Violence 
Prevention  and  Public  Awareness,  the  Attorney  General's  Elderly  Protection  Project,  and  the 
Department  of  Probation's  Domestic  Violence  Training.  With  the  assistance  of  an  NIJ  technical 
advisory  team,  the  two  programs  selected  to  provide  the  information  herein  were  the  Cambridge 
Police  Department's  domestic  violence  program  (Operation  Safe  Home)  and  the  Attorney 
General's  project  focusing  on  elder  abuse.  The  study  built  on  and  is  more  comprehensive  than  the 
internal  evaluations  incorporated  into  the  projects  themselves. 
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An  Overview  of  Each  Project  Evaluated 

Cambridge  Police  Department  Operation  Safe  Home 

The  project  began  on  July  1,  1993.  This  program  planned  to  develop  an  integrated 
response  system  between  Cambridge  police  and  community  organizations.  The  system  included  a 
computerized  database  for  monitoring  domestic  violence  and  sexual  assault  cases  Project 
components  included  the  deployment  of  personal  alarm  systems,  implementation  of  domestic 
violence  investigation  teams,  and  the  development  of  domestic  violence  offender  profiles  The 
project  also  worked  with  the  city's  Commission  on  the  Status  of  Women  in  a  publicity  campaign 
on  domestic  violence. 

The  two  specific  objectives  were: 

1 .  Utilization  of  a  standard  approach  by  officers  of  the  Cambridge  Police 
Department;  and 

2.  Linkage  of  the  police  department  with  community  based  advocacy  for  victims 
and  abusers. 

Evaluation  of  the  project  by  the  Cambridge  police  analyzed  patterns  of  and  responses  to 
domestic  violence  using  a  computerized  database.  It  cross-referenced  incidents  with  repetition  of 
offenses,  restraining  order  violations,  presence  of  children,  and  use  of  weapons.  The  offender 
profile  developed  was  used  to  identify  risk  factors  for  repeat  offenses  and  the  offender's  potential 
for  dangerous  actions.  This  project  would  imply  the  need  to  develop  behavioral  outcome 
measures. 

Attorney  General's  Training  Program  to  Combat  Elder  Abuse 

The  Elderly  Protection  Project's,  "Interdisciplinary  Training  Program  to  Combat  Elder 
Abuse,"  was  administered  by  the  Office  of  the  Attorney  General  for  the  Commonwealth  and 
began  on  July  1,  1992. 

The  project  provided  multi-disciplinary  training  for  over  400  police  officers.  Training 
materials  were  developed  and  disseminated  to  the  District  Attorney's  Offices  and  to  the 
Massachusetts  Criminal  Justice  Training  Council.  Model  protocols  for  police  responses  to 
incidents  of  elder  abuse  were  also  developed. 

The  specific  objectives  were  to: 

1 .  Review  existing  elder  abuse  training  programs  and  conduct  a  state-wide 
survey  to  assess  the  need  for  elder  abuse  training  to  develop  a  curriculum  for 
such  training; 

2.  Develop  model  protocols  for  police  and  protective  services  personnel; 

3 .  Provide  training  materials  for  police  officers  and  protective  service  personnel; 

4.  Devise  and  implement  a  prototype  training  program  at  five  locations  across 
the  state; 

5.  Organize  and  present  the  pilot  training  programs; 

6.  Assess  the  effectiveness  of  the  pilot  training  programs  through  the  use  of  a 
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mandatory  evaluation  instrument  for  conference  participants; 

7.  Utilize  the  evaluations  of  the  pilot  training  programs  to  finalize  the  model 
protocols  and  training  curriculum; 

8.  Disseminate  the  training  materials  through  the  use  of  a  final  report;  and 

9.  Coordinate  the  development  of  a  plan  for  subsequent  training  by  the 
Massachusetts  Criminal  Justice  Training  Council. 

Evaluation  of  the  curriculum  and  protocols  by  the  Attorney  General's  Office  used 
qualitative  assessments  of  police  chiefs  and  officials  in  concerned  state  agencies.  For  training 
sessions,  an  evaluation  form  was  completed  by  participants  on  their  reactions  to  the  training 
materials  and  the  training  sessions.  Evaluation  forms  were  also  sent  to  training  participants 
between  six  and  twelve  months  after  the  training.  These  follow-up  instruments  asked  what  aspects 
of  the  training  were  most  useful  to  police  officers  and  what  additional  materials  would  have  been 
beneficial.  Thus  attempts  were  made  to  assess  self-reported  changes  in  attitude  as  well  as 
behavior. 


DATA  COLLECTION 

NIJ  Technical  Assistance 

The  National  Institute  of  Justice  (NIJ)  provided  a  technical  assistance  team  to  help  the 
project.  Technical  assistance  helped  determine  which  domestic  violence  training  programs  to 
include  in  the  evaluation  and  what  research  methods  were  used.  Technical  assistance  in  the 
design  and  plans  for  implementation  of  training  programs  were  especially  desired  since  there  are 
long  standing  difficult  obstacles  encountered  when  evaluating  such  programs.  Information 
provided  by  the  technical  assistance  team  enhanced  the  capacity  of  the  Massachusetts  SAC  to 
evaluate  these  types  of  programs  even  after  this  particular  project  was  completed. 

The  strategy  for  data  collection  was  the  primary  area  in  which  technical  assistance  was 
requested.  The  two  strategies  used  most  often  for  evaluating  training  programs  are  pre-post 
designs  and  post-test  only  designs.  The  pre-post  designs  are  sometimes  augmented  by 
comparison  groups  that  do  not  undergo  training.  The  application  of  pre-post  design  for  domestic 
violence  training  has  looked  at  knowledge  and  behavior  of  criminal  justice  personnel  before  and 
after  training.  Knowledge  measures  with  this  design  have  focused  mainly  on  knowledge  of 
existing  statutes,  regulations,  and  recommended  procedures.  Behavioral  measures  have  looked  at 
provision  of  court  protection  orders,  arrest  practices,  and  other  police  responses.  Post-test  only 
designs  often  rely  upon  retrospective  information  on  perceived  change  from  prior  actions  and 
knowledge  and  speculation  on  future  actions  or  knowledge  use.  For  example,  questionnaires  at 
the  end  of  a  training  session  may  ask  what  police  officers  might  do  differently  as  a  result  of  the 
training  or  what  information  they  found  "helpful"  or  "useful"  (without  specification  of  helpful  in 
what  way  or  useful  for  what  purpose). 
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Design 

A  number  of  difficulties  arise  with  these  designs.  The  pre-post  design  confounds  historical 
and  developmental  changes  with  the  training  (Campbell  &  Cook,  1982).  If  the  pre-test  is  given 
immediately  at  the  beginning  of  the  training  to  reduce  bias  from  history  and  development,  subjects 
may  recall  material  at  the  time  of  the  post -test,  creating  an  instrumentation  effect  Such  training 
programs  are  often  voluntary,  allowing  for  sample  selectivity  to  alter  the  findings  (i.e.,  selecting 
criminal  justice  personnel  receptive  to  change  and  overstating  what  the  program  effect  would  be 
for  personnel  resistant  to  change).  The  use  of  comparison  groups  provides  some  improvement, 
but  these  comparison  groups  are  often  not  strictly  equivalent.  The  voluntary  nature  of  some 
training  means  the  groups  are  not  randomly  assigned  and,  consequently,  require  statistical  control 
procedures  to  be  used  to  remove  some  of  the  pre-group  differences.  Even  with  random 
assignment,  there  is  a  significant  risk  of  contamination  of  the  experiment.  Those  taking  the 
training  may  communicate  what  they  have  learned  (if  anything)  to  their  co-workers,  including 
personnel  who  are  in  comparison  groups.  We  have  observed  that  the  transmission  of  information 
between  departments  is  quite  high  when  officers  define  it  as  important  or  interesting.  This  implies 
that  the  greater  the  effect  of  the  training  program,  the  more  likely  information  from  the  program 
will  contaminate  the  comparison  group. 

The  weakest  form  of  the  post-test  only  design  typically  asks  those  who  have  completed  a 
training  program  to  use  their  recollection  of  the  training  to  evaluate  how  much  they  have  learned 
and  changed  or  use  speculation  to  project  how  much  they  think  they  will  act  differently  in  the 
future.  The  strongest  form  of  post-test  only  design  incorporates  random  assignment  into 
treatment  and  control  groups.  This  strategy,  however,  requires  that  the  subjects  not  have  the 
opportunity  to  communicate  with  one  another  before  the  post-test  is  given.  It  also  means  that 
subjects  cannot  volunteer  for  or  decline  the  training,  which  may  increase  levels  of  resentment  at 
not  being  given  a  choice.  This  may  attenuate  the  intervention  effects  of  the  treatment. 

Measurement 

Greater  clarity  is  needed  regarding  the  outcomes  to  be  measured  and  how  best  to  measure 
them.  Training  programs  may  alter  knowledge,  emotions,  attitudes,  or  behaviors—either 
intentionally  or  unintentionally.  Goal  statements  for  these  projects  may  mention  some  or  all  of 
these  targets  for  change. 

Instruments  to  measure  changes  resulting  from  training  are  highly  varied  and  sometimes 
quite  vague.  Poor  instruments  may  simply  ask  respondents  how  much  they  have  learned  from  the 
training,  how  much  they  have  enjoyed  the  course,  or  how  helpful  it  seemed  to  be.  Slightly  better 
instruments  may  represent  tests  of  knowledge  or  attitude  scales,  which  may  or  may  not  have  been 
examined  for  evidence  of  reliability  or  validity.  Even  behavioral  measures,  such  as  used  in  the 
evaluation  of  the  Massachusetts  mandatory  arrest  law  for  violators  of  court  protection  orders 
(Holmes  et  al.,  1993),  can  be  confounded  by  altering  the  behavior  of  others  who  are  affected  by 
changes  in  the  criminal  justice  officials.  For  example,  in  the  evaluation  of  the  impact  of  the 
mandatory  arrest  law,  the  number  of  arrests  increased,  but  a  substantial  group  of  cases  still  did  not 
result  in  arrest.  This  was,  in  part,  due  to  perpetrators  leaving  before  the  officers  arrived.  Whether 
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offenders  became  more  prone  to  flee  the  scene  before  the  officers  arrived,  as  a  result  of  the 
change  in  the  arrest  law,  could  not  be  determined  by  the  study,  but  it  was  a  possibility. 

Policy  Advisory  Panel 

An  advisory  committee  was  formed  to  provide  specific  input  regarding  the  substantive 
focus  of  the  research  and  utilization  of  the  findings.  This  advisory  committee  included  the 
domestic  violence  policy  specialist  for  the  Massachusetts  Committee  on  Criminal  Justice,  the 
domestic  violence  training  coordinator  for  the  Criminal  Justice  Training  Council  for  the 
Commonwealth  of  Massachusetts,  and  a  criminal  justice  researcher  from  the  University  of 
Massachusetts  at  Boston  who  had  experience  in  the  study  of  domestic  violence.  These  individuals 
had  been  involved  in  a  variety  of  roles  in  the  policy  making  process. 

EXAMINING  THE  TRAINING  PROJECTS 

Two  Approaches  to  the  Study 

The  study  used  a  combination  of  experimental  and  quasi-experimental  methods  to  examine 
impacts  of  domestic  violence  training  programs  in  the  Commonwealth.  Some  of  the  individuals  in 
the  training  programs  were  assigned  by  random  assignment,  some  were  volunteers,  and  some 
were  individuals  identified  as  priorities  for  training  and  were  required  to  participate  by  their 
respective  agencies.  The  mixed  design  addressed  two  important  issues.  An  experimental 
component  allowed  for  more  rigorous  assessment  of  the  project  outcomes.  A  quasi-experimental 
component  allowed  individuals  for  whom  training  was  identified  as  a  priority  to  still  receive 
expedited  training.  The  use  of  volunteers  also  allowed  exarnining  the  effects  of  the  program  on  a 
large  segment  of  the  most  common  recipients  of  such  training. 

The  priority  and  volunteer  subjects  in  the  training  programs  were  identified  by  the 
agencies  doing  the  training.  They  entered  the  program  by  the  procedures  designated  by  the 
agency  in  its  original  funding  proposal.  This  minimized  testing  effects  of  the  evaluation  on  the 
training  program  and  allowed  it  to  more  closely  resemble  the  "natural  setting"  in  which  such 
training  programs  take  place.  This  also  reduced  disruption  of  the  program  and  burden  to  the 
agency  resulting  from  the  evaluation. 

The  project  tried  to  assign  some  individuals  to  the  training  using  a  random  assignment 
process  as  part  of  the  requirements  for  evaluation  of  the  programs.  Procedures  for  assignment 
were  discussed  with  technical  assistance  from  the  National  Institute  of  Justice  team  to  insure  a 
process  that  was  workable  and  that  would  promote  acceptance  by  the  agencies.  Both  agencies 
agreed  to  cooperate  with  the  evaluation,  but  there  was  a  significant  need  to  devise  procedures  that 
were  perceived  by  them  as  fair,  equitable,  and  not  contrary  to  agency  or  programmatic  objectives. 
Obstacles  raised  by  these  challenges  undermined  the  ability  to  randomly  assign  training  subjects. 

Because  of  the  expected  experimental  component,  data  collection  was  only  planned  post- 
intervention.  This  also  reduced  respondent  and  agency  burdens  and  more  closely  simulated  a 
typical  assessment  and  monitoring  process  for  such  programs.  The  obstacles  that  prevented 
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complete  random  assignment,  however,  did  not  mean  we  could  switch  to  a  pre-  post-test  or 
longitudinal  design.  Advantages  of  a  post-test  only  data  collection  included  less  disruption  of  the 
programs,  allowing  the  project  access,  and  the  accommodation  of  available  resources  These 
points  were  great  enough  that  they  prevented  use  of  alternative  designs  that  could  better  cope 
with  non-random  assignment.  As  noted  earlier,  when  an  experimental  protocol  breaks  down,  an 
experimental  design  is  difficult  to  reconfigure. 

Measurement 

Refinement  of  measurement  tools  for  evaluating  these  programs  was  an  important  element 
of  this  study.  Focus  groups  with  the  advisory  committee  and  representatives  of  the  programs 
clarified  the  extent  to  which  measures  of  knowledge,  beliefs,  attitudes,  feelings,  and  behaviors 
were  needed.  The  current  more  general  measures  were  also  used,  despite  their  obvious 
deficiencies,  to  allow  contrasting  results  between  them  and  more  refined  measures.  This  allowed 
the  current  study  to  build  on  other  research  and  identify  implications  for  other  projects  using  the 
more  general  measures. 

Evaluation  Results 

The  project  produced  two  special  reports  (Kohl,  Brensilber  &  Holmes,  1995;  Holmes  & 
Headley,  1995).  These  special  reports  detailed  the  evaluation  findings  for  the  two  training 
projects.  They  discussed  the  evaluation  methodology,  strategies  for  enhancing  a  state's  evaluative 
capacity,  and  integration  of  findings  into  the  planning  process.  The  special  reports  summarize 
training  needs  identified  by  the  several  programs  evaluated,  strategies  that  met  those  needs, 
obstacles  to  successful  training,  and  methods  for  reducing  those  obstacles.  They  also 
recommended  modifications  to  existing  training  programs  to  make  them  more  effective.  The  two 
special  reports  are  included  in  the  appendices  to  this  report. 

All  data  sets  created  as  part  of  this  evaluation  project  will  be  provided  to  NIJ  in  machine 
readable  form  with  documentation.  They  will  conform  to  the  Data  Resources  Program  Handbook 
requirements  of  the  National  Institute  of  Justice  and  are  available  to  other  researchers  for 
secondary  analysis.  Copies  of  the  data  and  the  evaluation  instruments  are  also  available  from  the 
Massachusetts  Statistical  Analysis  Center  in  the  Executive  Office  of  Public  Safety  Programs 
Division. 

Training  Results 

Both  training  programs  attempted  to  modify  knowledge,  attitudes,  and  behavior  of  the 
trainees.  The  specialized  reports  on  each  project  clearly  indicate  that  knowledge  transfer  did 
occur.  Officers  and  other  trainees  became  more  aware  of  laws  and  regulations  governing  their 
actions  in  the  area  of  training.  They  learned  about  background  circumstances  that  may  have 
contributed  to  the  situation  to  which  they  were  responding  (such  as  Alzheimer's  Disease 
contributing  to  elder  abuse  and  sexism  contributing  to  domestic  violence).  Many  of  the  officers 
reported  changes  in  their  attitudes  regarding  domestic  violence  as  a  result  of  the  training 
experience.  Similarly,  participants  indicated  their  intent  to  change  their  behavior. 
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The  enhanced  evaluation  procedures  indicated  the  results  of  behavioral  change  The 
number  of  elder  abuse  reports  filed  by  police  to  elder  protective  service  agencies  increased  after 
the  training.  The  number  of  domestic  violence  incidents  identified  by  police  also  increased.  It  is 
less  clear  whether  these  behavioral  changes  persisted  or  faded  across  time.  The  short  duration  of 
follow-up  did  not  allow  determining  whether  this  was  a  seasonal  fluctuation,  a  random  variation, 
or  a  meaningful  drop.  Even  if  there  was  some  fading  of  behavior,  however,  the  level  of  reporting 
for  both  projects  remained  considerably  higher  at  the  end  of  the  study  than  at  the  beginning. 
Whether  the  behavioral  change  will  continue,  fade,  or  increase  will  likely  depend  on  whether  there 
is  follow-up  reinforcing  these  changes. 

Qualitative  information  from  the  projects  also  suggested  behavioral  changes  occurred. 
Many  officers  reported  changing  their  behavior  in  follow-up  interviews.  Service  agencies 
received  an  increase  in  referrals  and  greater  cooperation  from  the  law  enforcement  community. 
Project  coordinators  also  reported  changes  in  officer's  behavior.  The  qualitative  reports  from 
these  diverse  sources  presented  a  picture  consistent  with  the  quantitative  data. 

RECOMMENDATIONS 

Improved  Training 

The  trainers  employed  tactics  to  reduce  obstacles  to  changing  knowledge,  attitudes,  and 
behavior.  These  tactics  improved  knowledge  delivery,  attitude  change,  and  behavioral 
modification. 

To  strengthen  knowledge  delivery  they  used 

0        A  planned  curriculum;  and 

0        Validated  information  tests. 
Attitude  change  appeared  more  likely  when  participants  were 

0        Encouraged  to  express  their  opinions  freely,  especially  differing  ones;  and 

0        Exposed  to  people  with  direct  experience  in  working  with  the  victims  or  to 
victims  themselves. 

Learning  new  behaviors  was  encouraged  by 

0        Role  playing  the  desired  behaviors. 

These  recommendations  for  improved  training  are  not  new  innovations.  In  the  press  of  program 
development  and  operation,  however,  they  are  easy  to  bypass  or  overlook.  Trainers  of  domestic 
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violence  programs  need  to  periodically  step  back  from  the  process  and  review  what  they  are  doing 
and  assess  whether  these  tactics  may  help  them  improve  their  training. 

Improved  Evaluations 

Some  of  the  major  obstacles  to  evaluation  were  created  by  the  speed  with  which  the 
training  was  implemented.  It  was  difficult  at  times  for  the  evaluators  to  keep  up  with  new 
developments  in  the  projects.  Several  tactics  are  suggested  for  dealing  with  these  problems. 

0        Fast  tracking  the  evaluation; 

0        Building  on  existing  instruments;  and 

0        Employing  utilization  of  focused  techniques. 

Fast  tracking  the  evaluation  means  that  evaluators  have  to  plan  ahead  for  alternative 
contingencies.  Change  in  these  projects  is  inevitable.  Alternative  adaptations  of  design, 
measures,  and  data  collection  should  be  started  before  some  of  these  changes  occur.  This  may 
seem  a  waste  of  resources  initially,  but  an  entire  evaluation  can  be  compromised  if  some  changes 
are  not  responded  to  quickly  and  appropriately. 

This  recommendation  may  seem  at  odds  with  use  of  complex,  yet  powerful,  experimental 
designs.  An  effective  experiment  needs  control  over  the  intervention  and  consistent 
implementation.  On  the  other  hand,  if  an  experimenter  does  not  anticipate  and  plan  for  potential 
threats  to  the  experimental  protocol,  the  entire  experiment  can  collapse.  The  breakdown  of 
random  assignment  in  this  evaluation  has  shown  that  potential.  This  recommendation  does  not 
oppose  experiments.  It  does  say  that  the  experiments  must  be  able  to  deal  with  a  potentially 
changing  environment.  Even  when  experiments  have  a  breakdown  of  protocol,  they  can  still  be 
designed  to  produce  useful  information. 

Considerable  time  was  devoted  to  measurement  issues.  Using  existing  instruments  is 
perfectly  fine,  as  long  as  the  authors  are  given  credit.  Indeed,  the  findings  become  more  powerful 
because  they  can  be  compared  to  prior  studies  that  used  the  borrowed  instruments.  Using  other's 
measures  also  allows  developing  a  stock  of  instruments  that  can  be  rapidly  adapted  to  new 
circumstances.  It  supports  development  of  fast  tracking  procedures. 

Ever  since  Michael  Patton  published  Utilization  Focused  Evaluation  (1976)  there  has  been 
support  for  working  with  stakeholders  to  identify  how  the  evaluation  may  meet  their  needs  for 
information.  Some  positive  feedback  was  given  the  evaluation  for  information  it  provided,  but  it 
was  not  entirely  successful  in  using  these  tactics.  A  major  problem  with  the  police  training 
evaluation  derived  from  the  difficulty  in  addressing  the  diverse  information  needs  of  the  training 
project.  It  is  a  challenge  to  employ  these  tactics,  but  their  benefits  can  be  great. 
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